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Complete 


The  Conditionality  of  Matrices 


by 

D.  K.  Faddeev 

A  non- singular  matrix  A  is  "well  conditioned"  if  the  solu¬ 
tion  of  the  system  of  linear  equations  Ax  =  b  is  stable,  i.e. ,  if 
it  varies  little  with  a  small  change  in  the  elements  of  matrix  A 
and  in  the  column  b  of  the  free  terms.  Some  quantitative  features 
of  conditionality  are  known  in  literature  -  the  so-called  condition 
numbers.  A.  Tiuring  [1]  proposed  to  use  the  N-number,  equal  to 

\  N(A)N(A"‘  ),  where  N(A)  *  V  SpA*A,  as  the  degree  of  condition- 
n 

ality,  A*  being  a  conjugate  and  transposed  matrix  from  A,  and  n- 

the  order  of  the  matrix  A.  A  stricter  but  easier  computed  number 

M  equal  to  1_  M(A)M(A"1  ),  where  M(A)  =  n  max  |a. .  |,  can  be  the 
n  1 J 

degree  of  conditionality.  It  is  known  that  N(AKM(A)  so  that  the 

condition  number  N  does  not  exceed  M.  The  least  value  of  N  is 
1,  and  this  is  reached  for  the  "most  conditioned"  single  matrix. 

High  values  of  the  condition  numbers  characterizes  a  poorly  condi¬ 
tioned  matrix. 

The  N-number  as  well  as  the  M-number  are  products  of  cer¬ 
tain  norms  [N(A)  and  M(A)]  of  the  matrix  A  and  its  reciprocal,  with 

the  normalising  factor _1  •  Starting  from  other  norms  we  can  con- 

“n 

struct  several  other  condition  numbers.  The  most  natural  is  the  H- 
number,  equal  to  MIKU  where  |A|8-IMX|i(s|nVh.  IS  i»  the 
largest  eigenvalue  of  the  matrix  A*A. 
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It  is  easy  to  see  that  \^A  w^ere  *•  the  smallest 

eigenvalue  of  the  matrix  A*A.  Thus  the  H-number  of  the  condition¬ 
ality  is\H^l—  •  For  symmetrical  matrices,  it  agrees  with  the  P- 
V^n  k  | 

number  of  Todd  [2],  which  is  equal  to  mAX  *  1 j  ,  when  -are 

min  I  I 

eigenvalues  of  the  matrix  A.  11 

We  will  explain  the  sense  of  the  condition  numbers  N  and 
H  of  the  matrix  A,  by  starting  from  the  following  roughly  approxi¬ 
mate  probability  scheme. 

We  will  examine  the  effect  of  the  change  in  the  elements  of 
matrix  A  on  the  change  in  the  solution  of  the  corresponding  linear 
system  Ax  *  b,  where  b  is  a  vector,  which  will  be  considered  accu¬ 
rate  here. 

Let  us  assume  the  elements  of  the  matrix  A  are  real  independ¬ 
ent  random  values  with  a  standard  deviation  or 2  and  average  values 

V 

Let  (ttjj)*1  =  a^  .  Then 


and 


so  that#  dx^  can  be  considered  linear  combinations  of  the  independent 

values  of  da..  .  Let  us  find  the  second  moments  for  the  vector 
U  * 


<:> 


i.  c. ,  let  ue  compute  £  .  dx^dxj 


*M.  O.  is  Russian  notation  for  the  Western  £. 


We  have 
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u.  o.  dx*dx i  =  u.  o.  2  2*"*'-**'*1'****"' 

*J  m 

Due  to  the  independence  of  da. .  and  da 

ij  pq 

m.  0.  i*dd*n  =  0,  if  i+p,  j+q. 

Furthermore,  according  to  assumption 

M.  O. 

therefore 

wK.r.  I *.-$%*.  i.  «K.  .*•»«.  of  fh.  kfh  row  «d  of  co.wno  «  of 
the  matrix  4-1 M“*)*  =*  (AM)*"4  •  Thus,  in  particular. 

M.0.  \d*f  ■>*.«.  (*{+••  +  + 

m(4i /ww <iior 

x«+---+«B-^w(4-W-  j.— - - 

where  n  (A)  is  the  N-number  of  conditionality.  Extracting  the  root, 
we  get 


,,wT^T‘ 

By  considering  that  —  N(A)  is  the  mean  square  value  of  the 
n 

element  of  matrix  A,  and  «r  is  the  mean  square  of  the  errors  in  the 
elements  of  the  matrix  (equal  to  the  mean  square  error  of  r  of  each 
element  of  o^),  we  find  that  the  condition  number  N  indicates  how 
many  times  the  ratio  of  the  mean  square  error  of  the  unknown  to 
the  mean  square  of  the  unknowns  themselves  exceeds  the  ratio  of  the 
mean  square  error  of  the  coefficients  of  the  system  to  the  mean  square 
of  the  coefficients  themselves. 
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In  [1],  Tiuring  gives  (verbally)  an  estimate 

|/-LyLpf  <N(A)N  (A-*)  -J— =  H*  (A) 

“  ■*“  -i  MUt  —  M 


MUD 


is  valid  without  assuming  the 


ij 


where 

equality  of  all  <r . 

Let  us  turn  to  the  formula 

M.  0. 

Assume  the  vector  dx,  for  which  the  moments  are  given  in  the  last 

A  4 

formula,  has  the  multivarying  distribution  density  4>  r|f|  , 

(W|*nT 

where  Ais  the  absolute  value  of  the  determinant  of  matrix  A, 


Q(t, 


tR)  -  the  quadratic  form  with  matrix  A*A. 


Because  of  thi^the  condition  number  H  of  the  matrix  A, 
which  is  equal  to  ft  gives  the  ratio  between  the  major  semi- 
axis  of  the  ellipsoid  of  concentration  Q(tt  ,  • .  • ,  tn)  *  c  and  its  minor 
semi -axis;  i.  e. ,  the  H- number  characterises  the  lengthiness  of  the 
ellipsoid  of  concentration. 

Therefore,  for  a  poorly  conditioned  matrix,  solution  of  the 
linear  system  is  least  specific  toward  the  long  semi-axes  of  the 

ellipsoid  Q(xt . .  xn)  *  c,  i.  e. ,  toward  the  eigenvectors  of 

the  matrix  A+A  belonging  to  the  small  eigenvalues. 

Let  us  note  that  when  the  free  terms  are  also  independent, 
random  values  having  a  uniform,  small  dispersion,  the  second 
moments  of  the  components  of  the  solution  differ  only  by  a  multi¬ 
plicative  factor  from  the  corresponding  moments,  which  were  cal¬ 
culated  assuming  constancy  of  the  free  terms. 
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Namely: 


where  <r  *  is  the  dispersion  of  each  of  the  free  terms  of  the  system. 

We  now  turn  to  an  examination  of  the  eigenvalues  and  eigen¬ 
vectors.  We  will  assume,  as  before,  the  matrix  elements  are  random 
values  with  the  mean  values  and  uniform  dispersions  <r .  The  eigen¬ 
values  of  the  matrix  (o^)  will  be  considered  as  natural  and  mutually 
different. 

From  the  equality 

we  conclude 

I**)  «« + 

By  scalar  multiplication  of  this  equality  by  the  eigenvector 
of  the  transposed  matrix,  we  will  get 

0 id) a*  •ri+(Atdui).  vj> 

whence 

Without  disturbing  the  generality,  we  can  consider  (u^  v^)  *  I, 
when  ¥/  »«•  (*•- 


When  i  »  j,  we  get 


whence 

and 
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In  particular, 


„.o.  =•*(*„  «,)(•„  *4)»  w£r5r* 

The  coefficient  — p^L— — j-  has  a  clear  geometrical  sense.  It 
characterizes  the  deviation  of  the  eigenvector  from  the  perpendicu¬ 
lar  v.  to  the  space  spanned  by  the  remaining  eigenvalues.  Therefore, 


the  number 


vector  u.. 
1 


can  be  called  the  bending  coefficient  of  the  eigen- 


Thus,  the  variance  of  the  eigenvalue  is  «r  2 ,  multiplied  by  the 
square  of  the  bending  coefficient. 

If  the  matrix  (o„)  i®  symmetric,  the  result  is  obtained  quite 
simply  i.  e.  ,  H.  9.  and  Um  so  that  in  this  case,  the 

eigenvalues  will  be  independent  random  values  with  variance  <r*. 

This  conclusion  is  drawn  under  the  unlikely  assumption  that 
all  the  matrix  elements  are  independent  as  well  as  symmetric  with 
respect  to  the  main  diagonal. 

However,  we  can  consider  that  the  result  is  valid,  when  the 
matrix  (and  not  only  its  average  value)  is  considered  symmetrical, 
when  the  variance  of  the  diagonal  elements  is  taken  as  and  that 
of  the  non-diagonal  as  ±  <r 

The  law  of  concentration  of  the  components  of  eigenvectors 
is,  naturally,  quite  awkward,  even  in  the  examined  simplified 
system.  Without  mentioning  formulas,  we  note  that  the  bending 
coefficient  as  well  as  the  proximity  of  the  eigenvalues  are  affected 
by  the  stability  of  the  eigenvectors. 
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